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Prefatory Note 

This paper was presented at the American Psychological 
Association Convention in Miami Beach, Florida, September 1070, as 
part of a symposium entitled, 'The Implications of Aptitude level 
and Instructional System for Q kiK Acqui$it;* f ' and Job Performance." 
The research reported in this paper was performed at HumRRO 
Division No, 3, Presidio of Monterey, California, under Department 
of the Army contract with the Human Resources Research Organiza- 
tion as part of Work Unit UTIUYY, Study of Soldie's in Lower 
Mental Categories: Job Performance and the Identification of 
Potentially Successful and Potentially Unsuccessful Men. 
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PERFORMANCE IN FOUR JOBS: THE ROLE OF 
MENTAL ABILITY AND EXPERIENCE 

Robert Vineberg and Elaine N. Taylor 



The problem of the lower aptitude man in the military, what he can do and what he 
can’t do, is a perennial one. While experience has provided ample evidence that many 
men with low aptitude tost scores can perform effectively, there is no systematic 
information about how well they really do. The data used are part of the HumRRO 
Research done under Work Unit UTILITY to provide such information. The research was 
designed to compare the job performance of men at different ability levels and to map 
the manner in which their performance changes as a function of the length of time they 
have been in their jobs. Since there is evidence that many low aptitude men do perform 
effectively, the research is also aimed at identifying those personal and background 
characteristics that might distinguish more effective marginal individuals. 'This paper, 
however, will be concerned with data bearing on only the first objective— mapping of 
performance as a function of the Armed Forces Qualification Test (AFQT) grouping and 
job experience. 

The jobs we studied were Armor Crewman, Vehicle Repairman, Supply Specialist, 
and Cock. The Army jobs chosen were highly populated and with numbers of low 
aptitude men. These particular jobs were also selected because they covered a fairly broad 
range of job types ar.d task complexity and because, with the exception of the Armor 
Crewman, each had a civilian counterpart. 

The Armor Crewman job may be considered machine*a>cendant in that it principally 
consists of sequences of interaction between a man, a tank, and its weapon systems, with 
most tasks being largely procedural. The Repalnan job is a maintenance job that 
requires diagnostic and interpretive skills. The Supply Specialist job is clerical, calling 
primarily for the coordination and recording of information. The Cook job calls for 
preparation of food in accordance with a master menu and detailed recipes. Recognition 
of standards and precision in meeting then are primary characteristics of this job. 

Job incumbents were tested in combat divisions in the United States and in 
Germany. Within tech division pairs of men were selected for testing; one man with an 
AFQT or percentile score between 1 and 30 and one man with a >core between 31 and 
99. Pair members were carefully matched for the amount of time they had been in their 
jobs. 1 Particular care was taken to ensure that all men in the sample were working on a 
continuous daily basis in their jobs. A total of approximately 399 men or 195 pairs were 
tested in each job. 

We measured performance using job sample tests, conventional multiple choice 
knowledge tests, and supervisor ratings. As is frequently the case, supervisor ratings failed 
to be discriminating enough to warrant further discussion. Our primary criteria were the 
job sample tests. We believe ours is the only study that has undertake!. such extensive 
ar.d direct measurement of performance. Each individually administered job sample test 
took from 3t4 to 5 hours, depending on the job. 



o 

ERIC 



1 In et:h job tome r.en were studied who possessed up to 20 year* of job experience. 
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Each job sample test was comprised of a number of subtests. To make testing 
realistic and to simulate job conditions with as much fidelity as possible, each subtest 
consisted of the performance of a single entire task with a natural beginning and ending. 
Tasks were composed of a series of actions or steps that would ordinarily be performed 
as part of a single operation in the performance of a job. 

Subtest tasks or problems were introduced to subjects just, as they might ordinarily 
encounter them. For example, one of the Repairman tasks involved the diagnosis and 
correction of a leaking oil seal in a fan shaft of a tank. The tank was prepared so that 
the oil seals leaked and the subject was told that oil was being thrown out of the tank’s 
grill doors. The Repairman was to locate the source of the oil leak and repair the 
malfunction. Figure 1 shews the Repairman working on the initial steps. 

General Vehicle Repairman's Test-Unbolting Fan Assembly 




Figure t 



In this problem there were 19 necessary steps. Subjects earned a point for the 
correct performance of each step. This scoring procedure was used throughout the 
testing. The 13 subtesti administered to Repairmen are shown in Figure 2 as an example 
of the job sample instruments used in all four job sample tests. 

Figures 3-6 show the pe formance of men in different AFQT groups in the four 
jobs. Performance is plotted at 1-9 months on the job, 10 24 months, 26-60 months, and 
more than 60 month*. Figure 3 shows results for Armor Crewman. 

First, scores for each AFQT level go up with time in the job. While performance 
increases in each AFQT group with time, the groups maintain their positions relative to 
each other through the 26 to 60 monih period on the job It is not until men have been 
in their jobs for more than five years that we begin to see some evidence of convergence 
of the different groups. At this point men are averaging about 80% on the test which is a 
rise of about 30% from the first months on the job. 

The Cook rate of growth is gradual (see Figure 4). Performance appears to start at a 
relatively higher level, probably because job procedures are described in cookbooks which 
were used during testing. 
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Repairman's Test 



SUBTEST 



NUMBER 
OF STEPS 



SHIFT LINKAGE ADJUSTMENT, M60A1 TANK . 11 

TRANSMISSION SERVO BAND ADJUSTMENT, M60A1 TANK 13 

VOLTAGE REGULATOR ADJUSTING RHEOSTAT ADJUSTMENT, MS0A1 TANK ... 17 

HYDRAULIC BRAKE PEDAL ADJUSTMENT, M60A1 TANK 11 

OIL SEAL MALFUNCTION IN COOLING FAN VERTICAL DRIVE SHAFT, 

DETECTION AND REPLACEMENT, M60AI TANK 19 

IGNITION MALFUNCTION, FAULTY SPARK PLUG, DETECTION, M151 JEEP 9 

IGNITION TIMING, CHECK AND ADJUSTMENT, M151 JEEP 19 

CYLINDER COMPRESSION CHECK, M151 JEEP 13 

BATTERY HYDROMETER TEST. M151 JEEP 11 

BATTERY CONDITION TEST, M151 JEEP 13 

WHEELBEARING, MALFUNCTION, DETECTION AND ADJUSTMENT. 

M151 JEEP 13 

FUEL PUMP PRESSURE CHECK, M35A1/A2, M4VC TRUCK 10 

BLEEDING SERVICE BRAKE SYSTEM, M3SA1/A2, M49C TRUCK _J7 

176 



Figure 2 



Performance Test Score by AFQT and Months on Job: Armor Crewman 
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Performance Test Score by AFQT and Months on Job: Cook 
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The results for the Repairman job in Figure 6 show growth as in the other jobs and 
fairly clear convergence after five years. 

Finally, the Supply Specialist job shows generally the same trends. Here the upper 
three AFQT groups converge at five years but with the lowest group still behind. 

The general pattern of development is pretty much the same overall. During the first 
nine months on the jeo men get about half tne items on the test. After five years on the 
job growth occurs at all AFQT levels to a point at which men are getting about 
three-fourths of the test items and where there is some evidence of convergence. 

Convergence is saen most clearly by focusing on the highest and lowest aptitude 
groups. 

As seen in Table 1, the average difference between the very highest and lowest 
AFQT groups during the first nine months on the job is about 16%. Beyond five years on 
the job this difference decreased to about 6%. The lowest AFQT group has gained an 
average of 28% whereas the highest AFQT group, who of course perform better initially, 
gained about 16%. 

Two different analyses of covariance were run to test the effects of AFQT and job 
experience. In one, we compared 0-20, 21*30, and 31*99 AFQT groups and partialled out 
time on the iob and age. In the ether, we compared men with 1*9, 10-24, end 26 months 
on the job and partialled out AFQT and education. 

In ell jobs, overall F test* for AFQT differences were significant and individual 
comparisons between the three AFQT groups were also significant. 

For time on the job all F$ were significant, and out of 12 comparisons between pairs 
of means, 11 were significant. 
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Performance Test Score by AFQT and Months on Job: Repairman 
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Tabte 1 

Performance Test Score Range of Lowest 
and Highest AFQT Qroups 

(of 1-9 and 6J + Monf/u-on-Job £xp*rf*nc«) 







Month* 


on Job 






1-9 


61 + 




AFQT 

0-20 65-99 


Different • 


AFQT 

0-20 65-99 


Oifferenc* 


Armor Crswmsn 


39-57 


18 


76-81 


5 


Cooks 


50-62 


12 


73-79 


6 


Rtpofnrtn 


53-74 


21 


84-86 


2 


Supply Spoelolisls 


50-63 


13 


69-76 


7 




Moon Diffortfico - 16% 


Mton Difftreocs - 5% 
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Performance Test Score by AFQT and Months on Job: Supply Specialist 
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Figure 6 



i_ 

. 25-60 



i /■ 



61 + 



Although the amount of difference between AFQT or job experience groupings 
obviously depends on the particular groups selected for comparison, larger differences 
appear across the job experience dimension than across the AFQT dimension. Correla- 
tions also show a higher relationship between job experience and performance *han 
between aptitude and perform a tee. The average correlation between job experience and 
performance, with aptitude partialled out, was .60, The correlation between aptitude and 
performance, with job experience partialled out, was .36. 

While the graphs using means show dear separation between AFQT groupings out to 
at least five years, a great deal of overlap exists across AFQT groups. An alternate 
method of presenting data is to group subjects according to their position on the job 
sample distribution. In Figure 7 the percentages of men in the different AFQT groupings 
falling in the lowest, two middle, and highest quarters cf the Job Sample distribution are 
presented. The figure Is drawn for men with 1-18 months of job experience for all MOSs 
combined. 

Figure 7 reflects the general relationship between AFQT and job performance. Thus 
it Is seen that there is a greater probability that men at the lower end of the AFQT 
distribution will perform in the lowest quarter. This is mirrored at the high end of the 
distribution In which men are more likely to score in the highest quarter. The critical 
observation if , however, that a substantial proportion of men at all AFQT levels fall into 
each quarter of the job sample distribution. For example, 33% of the 0-20 AFQT group 
score in the upper half of the distribution whereas 26% of the 65-99 AFQT group score 
in the lower half of the distribution. In the first 18 months, the similarity of 
performance of the High Category IVs and Category Ills Is striking. A little less than half 
(44%) of the High Category IV group scored above the median, whereas a little more 



Distribution of AFQT Groups With 1*18 MOJ on Job Sample Tests 

(Fear MOSs Combined ) 
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Distribution of AFGT Groups With '<9*30 MOJ on Job Sample Tests 

( Four MOSs Combined-in Relation to Quarters of 1-18 MOJ $ub$imp<3) 
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Job Knowledge Test Score by AFQT and Months on Job: Armor Crewman 
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than half (56%) of the Category III group scored above the median. (The latter group has 
been historically acceptable for service.) 

Men v/ith between one and 18 months of job experience made up 67% of the 
UTILITY sample in the four MOSs reported on. The Army in general is heavily weighted 
with men in approximately the first two years of a first tour. Since, at any given time, 
most men wortang in .in MOS fall within the first year and a half of job experience, 
performance in this time segment approximates modirl performance in the Army. The 
data for men with longer job experience have been sorted into the quarters used for the 
data of the 1*18 me nth group to compare their performance. See Figures 8 and 9. 

The implications to be drawn from these figures are that with time, an increasing 
proportion of men at t'd AFQT levels appear in the upper ranges of the performance 
distribution. Thus, among men with 19-30 months of job experience, 69% of the Low 
Category IV group fall in the upper half of the distribution. In the 30+ months group, 
85% of the Low Category tV group fall in this upper half. 

In addition to the peiformance tests, we administered more conventional multiple 
choice job knowledge tests. In Figures 10-13 test scores are plotted for the same AFQT 
groups and job experience intervals that were used in the earlier graphs. 

The results for Armor Crewman are given in Figure 10. 

Job knowledge increases rather regularly in all AFQT groups with time. While the 
spread is It.:-, beyond five years than at the beginning, there is no clear evidence for 
convergence. 

Figure 11 shows that the Cook had growth at all AFQT levels with some crossover 
in the two lowest AFQT groups. Again, there was, however, still no evidence of 
convergence. 



Job Knowledge Tesl Score by AFQT and Months on Job: Cook 
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Job Knowledge Test Score by AFQT and Months on Job: Repairman 
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Figure 12 



The results for the Repairman test given in Figure 13 show a not dissimilar pattern. 

Finally, the job of Specialist Supply shows considerable growth in job knowledge 
and is the only job in which there is evidence of convergence beyond the five years on 
the job (see Figure 13). 

This may be a ceiling effect as the results of the test show that men get about 80% 
of the test by the last time interval, whereas in the test results for other jobs the average 
ranged between 62 and 75%. Because the Supply Specialist job primarily involves the 
processing of information rather than overt physical performance, the convent of the job 
knowledge and job samp’e tests in this speciality are necessarily more alike. It is not 
surprising that the shapes of their curves are similar. What is a little more difficult to 
explain is the seeming l*ck of convergence over time on the other jobs. Since we have 
seen performance curves generally come together with time, we might expect curves of 
the knowledge which is required to support performance to do the same. On the other 
hand, there is more information to be learned about a job *han can be adequately 
represented in the types of tasks one is able to sample in constructing a performance test. 
(This concerns information which is less frequently required and hence is reviewed on a 
less frequent basis and often need not be memorized since it is available in seme variety 
of job support.) 

While we included some infrequently appearing tasks in our job cample teste, our 
knowledge teste covered a relatively broader range of job Information. Acquisition of 
such information occurs in a more incidental manner over a longer span of time. With 
this kind of learning, it is not unreasonable that reparation between curves describing this 



Job Knowledge Test Score by AFQT and Months on Job: Supply Specialist 
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Figure 13 



process be maintained. We believe that this is probably the main reason for a lack of 
convergence in the knowledge curves over the time intervals we have plotted. 

The data for all four jobs are so similar that only one, Armor Crewman, is used to 
show how performance varies in tasks of differing difficulty and complexity with AFQT 
and time. (See Figure 14). Performance is shown on "easy,” "medium,” and "difficult” 
job sample subtests for men with A V QT scores between 1 and 30 ai.d between 31 and 
99. Easy, medium, and difficult were defined empirically on the basis of the combined 
performance of both AFQT groups at all levels of job experience. 

In all jobs, the general pattern was clear. Learning occurs at all levels of task 
difficulty for the two aptitude groups. Performance in easier tasks tends to reach an 
asymptote with increasing job experience but not in the moke difficult tasks. Not 
surprisingly, the separation between aptitude groups is least in the "easy” tasks. Differ- 
ences between AFQT groups become greater as difficulty increases. 

The clearest pattern of differences between the two aptitude groups as a function of 
task difficulty, without regard to time, can be seen in Table 2. Here since there were 
only five Cook subtests, the data were not grouped for these comparisons. The differ* 
ences between the low aptitude and high aptitude groups increase rather consistently as 
problem difficulty becomes greater. Combining the jobs, the average difference between 
groups is 6.8% on the "easy” subtest, 8.8% on the "medium” subtest, and 11.3% on the 
"difficult” subtest. 

Table 3 shows data comparing the performance of low and high aptitude groups on 
task elements ot differing complexity. To do this, each step in a subtest was analyzed 
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Easy, Medium, and Difficult Job Sample Test Scores 
by AFQT and Months on Job: Armor Crewman 




Figure 14 



Table 2 

Differences In Performance on Easy, Medium, 
and Difficult Job Sampto Tests 

(parctnf) 





Eoiy 


| Medium 


j Difficult 


Armor Crewman 








AFQT 31-99 


877 


74.0 


447 


AFQT 0-20 


82.4 


66.1 


33.9 


Difference 


5.3 


7.9 


10.8 


Rtpolrmon 


AFQT 31-99 


847 


73.0 


57.9 


AFQT 0-20 


77.1 


64.6 


43.4 


Difference 


7,6 


8.4 


14.5 


Supply Specialist 


AFQT 31-99 


73.0 


64.2 


54.1 


AFQT 0-20 


68.5 


54.1 


45.5 


Difference 


4.S 


10.1 


8.6 
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Table 3 



Performance of Two AFQT Gi^. js in Two Sets 
of Behavior Categories: Armor Crewman 

(percent) 



Categories 


AFQT 

31-99 


Difference 


AFQT 

0-20 


Difference 


AFQT 31-99 
minus 0-20 


N 


86.3 ^ 


5.5 


74.7 j 


1 

\ 3.0 


11.6 


nk 2 $ 


80.8 - 


• 13.7 


71.7 j 


\ 12.1 


9.1 


nk 2 


6 7 . 1 




59.6 J 




7.5 


*1 


71.3 1 


\ 0.1 


65.7 | 


\ 2.3 


5.6 


k } k 2 


71.2 | 


f 3.2 


63.4 | 


\ 3.8 


7.8 




68.0 | 


\ 5,6 


59.6 < 


► 10.6 


8.4 


K } K 2 K 2 


62.4 ' 




49.0 ’ 




13.4 



according to the degree of structure provided to a subject for accomplishing the step. The 
structuredness of a step depended on such characteristics as: 

(lj Whether the step had been named or identified (e.g., as part of the test 
instructions or in a manual). 

(2) Whether technical or enabling knowledge was required. 

(3) Whether the step involved conforming to a precise standard, and, if so, 
whether the standard had been specified for the subject. 

We identified 24 logicaJly possible categories to which subtest steps could be 
assigned. Using these categories, two mutually exclusive classification sets were developed. 
The first sot contained, regardless of other characteristics, steps that have been named or 
identified for the subject. The second set contained steps that had not been named but 
which had to be known by the subject. 

This table shows categories and mean scores in the categories which were used in the 
Armor Crewman job. We considered only categories which contained a reasonably large 
number of steps and which were represented in at least two of the jobs. The results for 
Armor Crewman are the only ones discussed since categorization of subtest steps was 
complicated in the other jobs by the possibility of electing to use a technical manual 
when she test was taken. No manuals were used in the Armor Crewman tests. 

In the uppper set of categories, N stands for the very simplest situation; the step has 
been named and can be carried out without any additional information. NK2S stands for 
a step that has been named but in addition requires some technical knowledge (Kg) and 
conformance to a precise standard which has been specified for the subject (S), NK2 
stands for a named step which requires technical knowledge. We believe that an NK2S 
step is easier than an NK2 because, though, an additional element is present, 
specifying the standard to be met in conjunction with actually naming the step provides 
additional information which se.-ves as a cue to the recall of K2. 

In the lower set, Kj is a simple step analagous to an N step but ont which has not 
been named for the subject. KyK2 is an unnamed step requiring technical knowledge. 
K1K2S is an unnamed step requiring technical knowledge and conformance to a specified 



standard. K 1 K 2 X 3 is an unnamed step requiring technical knowledge and conformance to 
an unspecified standard. 

Looking at the mean percent scores, several things should be noted: 

(1) Without exception, high aptitude men outperform low aptitude men in 
each behavior category. 

(2) Within each set, the level of performance decreases for lows and highs alike 
a* complexity increases. 

(3) In either set, for either group, decrements in performance increase in a 
pocitive manner at each successive level of complexity. 

(4) In named tasks, where a large proportion of high aptitude men perform 
successfully, decrements in performance with increasing complexity are 
relatively greater for high aptitude men. In unnamed tasks, where neither 
group begins at a very high level, low aptitude men show relatively greater 
decrements in performance with increasing complexity. The overall effect 
in named tasks is a decreasing difference between lows and highs as 
complexity increases. In unnamed tasks, performance differences between 
lows and highs increase with complexity. Thus, we seem to have a 
U-shaped relationship. On named tasks (which, other things being equal, are 
easier) differences between lows and highs decrease as complexity increases, 
but on the more demanding unnamed tasks, differences between lows and 
highs grow wider with complexity. 
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